API: Unready Targets Dump by ping-sun · Pull Request #12554 · envoyproxy/envoy

ping-sun · 2020-08-07T22:34:55Z

Description: Taking advantage of the new feature introduced in #12035, which introduced quick visibility for init managers to check unready targets, this pull request adds protobuf message for unready targets and enables admin to dump configs of unready targets. An example of config dump for listeners’ unready targets is given in this pull request.

Introduce InitDumpHandler with handlerInitDump method to help dump information of unready targets.
Add dumpUnreadyTargets function for init::manager.

The new proto introduced:

// Dumps of unready targets of envoy init managers. Envoy's admin fills this message with init managers,
// which provides the information of their unready targets.
message UnreadyTargetsDumps {
  // Message of unready targets information of an init manager.
  message UnreadyTargetsDump {
    // Name of the init manager. Example: "init_manager_xxx".
    string name = 1;

    // Names of unready targets of the init manager. Example: "target_xxx".
    repeated string target_names = 2;
  }

  // The dumps of unready targets of all init managers.
  repeated UnreadyTargetsDump unready_targets_dumps = 1;
}

Signed-off-by: ping sun <pingsun@google.com>

…view; targetHandle->name(); remove target_names_ vector Signed-off-by: pingsun <pingsun@google.com>

Signed-off-by: ping sun <pingsun@google.com>

Signed-off-by: pingsun <pingsun@google.com>

…y-unready-targets

Signed-off-by: pingsun <pingsun@google.com>

…II/envoy into init-manager-query-unready-targets

Signed-off-by: pingsun <pingsun@google.com>

htuch · 2020-09-02T00:06:05Z

api/envoy/admin/v3/config_dump.proto

 }
+
+// Envoy's admin fills this message with init managers, which provides the information of their unready targets.
+message UnreadyTargetsConfigDumpList {


Is this actually configuration? It's definitely implied by configuration when interpreted by Envoy, but I'm wondering if it belongs on a different endpoint. @envoyproxy/api-shepherds

Is this actually configuration? It's definitely implied by configuration when interpreted by Envoy, but I'm wondering if it belongs on a different endpoint. @envoyproxy/api-shepherds

I suppose it does not belong to an actual endpoint. This is just a protobuf message designed to show the unready targets in configs.

I think it's kind of weird to include as part of config dump, but I'll defer to what other @envoyproxy/api-shepherds reckon. @mattklein123 as well as he did the original config dump.

+1, can we spit this out somewhere else? This is not really config in the normal sense of the word. Maybe a new /init_manager or other endpoint?

got it. I will move it somewhere else, and create a new init_dump.proto

Sorry I'm late to this thread, I feel this should be part of config_dump, otherwise it will be hard to tell what config made the targets unready, querying two endpoints might end up in inconsistent (racy) dump.

Isn't this a problem that applies to the admin endpoint in general? If we query /config_dump and /clusters, we might see inconsistent output. Same goes for /listeners. Would it make sense to try solve this through some mechanism for batching, e.g. an API endpoint, rather than trying to atomically group things?

I feel like /config_dump or the potential is an inspection endpoint.
The configs/targets are stale as soon as these init manager status are on the wire.

rst has been updated.

…gets-config-dump

Signed-off-by: pingsun <pingsun@google.com>

ping-sun · 2020-09-04T06:08:24Z

I've removed unready targets from config dumps and introduced a new init_dump to display that information. @mattklein123 @htuch

lambdai · 2020-09-08T18:04:30Z

Could you update the commit message reflecting the new init dump endpoint ?
And also in docs/root/version_history/current.rst

Signed-off-by: pingsun <pingsun@google.com>

ping-sun · 2020-09-08T18:50:07Z

Could you update the commit message reflecting the new init dump endpoint ?
And also in docs/root/version_history/current.rst

Sure, Updated

Signed-off-by: pingsun <pingsun@google.com>

…gets-config-dump

Signed-off-by: pingsun <pingsun@google.com>

htuch

/lgtm api

htuch · 2020-09-09T16:04:28Z

@lizan @dio will leave the rest of the PR for you folks, API LGTM.

ping-sun · 2020-09-09T22:35:17Z

I noticed that config_dump has been sealed in a unique class ConfigDumpHandler, so I followed this behavior and moved my init_dump in a new lib as well.

Signed-off-by: pingsun <pingsun@google.com>

ping-sun added 30 commits July 9, 2020 23:50

init-manager-query-unready-targets

10249c2

Signed-off-by: ping sun <pingsun@google.com>

init-manager-query-unready-targets

1de879e

Signed-off-by: ping sun <pingsun@google.com>

init-manager-query-unready-targets

5a7c308

Signed-off-by: ping sun <pingsun@google.com>

update: unordered_map->flat_hash_map; const std::string->absl:string_…

562cc49

…view; targetHandle->name(); remove target_names_ vector Signed-off-by: pingsun <pingsun@google.com>

init-manager-query-unready-targets

a023f16

Signed-off-by: ping sun <pingsun@google.com>

init-manager-query-unready-targets

e7ac416

Signed-off-by: ping sun <pingsun@google.com>

add dumpUnreadyTargetsConfig; remove checkUnreadyTargets()

b7fb8d8

Signed-off-by: pingsun <pingsun@google.com>

add dumpUnreadyTargetsConfigs()

c20129d

Signed-off-by: pingsun <pingsun@google.com>

add dumpUnreadyTargetsConfigs()

6458f27

Signed-off-by: pingsun <pingsun@google.com>

add dumpUnreadyTargetsConfigs()

452a264

Signed-off-by: pingsun <pingsun@google.com>

Merge branch 'master' of github.com:envoyproxy/envoy into exp

dc362e2

add extra constructor

0b7b75f

Signed-off-by: pingsun <pingsun@google.com>

add extra constructor

80af9b5

Signed-off-by: pingsun <pingsun@google.com>

add WatcherImpl constructors

2653323

Signed-off-by: pingsun <pingsun@google.com>

remove useless string_view parameter

9e2263c

Signed-off-by: pingsun <pingsun@google.com>

remove usesless string_view parameter

6428f5e

Signed-off-by: pingsun <pingsun@google.com>

add warming/active listeners config dump

6dbadd5

Signed-off-by: pingsun <pingsun@google.com>

add warming/active listeners config dump

83a7435

Signed-off-by: pingsun <pingsun@google.com>

add warming/active listeners config dump

dc4a1d4

Signed-off-by: pingsun <pingsun@google.com>

Merge remote-tracking branch 'upstream/master' into init-manager-quer…

4539ee5

…y-unready-targets

update stats_integration_test memory usage

8cd7fd6

Signed-off-by: pingsun <pingsun@google.com>

update variable name and comments

4f7900d

Signed-off-by: pingsun <pingsun@google.com>

update mem usage stats

27b6ca5

Signed-off-by: pingsun <pingsun@google.com>

update mem usage stats

fd5cbf4

Signed-off-by: pingsun <pingsun@google.com>

Merge branch 'init-manager-query-unready-targets' of github.com:ASOPV…

ef32968

…II/envoy into init-manager-query-unready-targets

update mem usage stats

f4409df

Signed-off-by: pingsun <pingsun@google.com>

update comment

031f585

Signed-off-by: pingsun <pingsun@google.com>

update comment

c4bf9a5

Signed-off-by: pingsun <pingsun@google.com>

update comments

e8e66ff

Signed-off-by: pingsun <pingsun@google.com>

add comments about the usage of WatcherImpl's ctor

17ae201

Signed-off-by: pingsun <pingsun@google.com>

htuch reviewed Sep 2, 2020

View reviewed changes

ping-sun added 2 commits September 2, 2020 01:20

Merge branch 'master' of github.com:envoyproxy/envoy into unready-tar…

c0038fd

…gets-config-dump

Merge branch 'master' of github.com:envoyproxy/envoy into unready-tar…

064b5cc

…gets-config-dump

ping-sun requested a review from htuch September 3, 2020 04:47

merge conflict

8c8c1e4

Signed-off-by: pingsun <pingsun@google.com>

ping-sun dismissed stale reviews from lizan and lambdai via 8c8c1e4 September 3, 2020 22:40

repokitteh-read-only bot added the api label Sep 3, 2020

dumps for unready targets; introduce new init_dump

f4d523d

Signed-off-by: pingsun <pingsun@google.com>

ping-sun requested a review from mattklein123 September 4, 2020 02:23

ping-sun added 2 commits September 8, 2020 18:40

api change: init dump using '/init_dump' to dump unready targets

1313edd

Signed-off-by: pingsun <pingsun@google.com>

api change: init dump using '/init_dump' to dump unready targets

54ccd99

Signed-off-by: pingsun <pingsun@google.com>

ping-sun added 4 commits September 8, 2020 20:25

api change: init dump using '/init_dump' to dump unready targets

06f355c

Signed-off-by: pingsun <pingsun@google.com>

Merge branch 'master' of github.com:envoyproxy/envoy into unready-tar…

7001271

…gets-config-dump

merge conflict

5950e54

Signed-off-by: pingsun <pingsun@google.com>

api change: init dump using '/init_dump' to dump unready targets

1ce686e

Signed-off-by: pingsun <pingsun@google.com>

htuch reviewed Sep 9, 2020

View reviewed changes

repokitteh-read-only bot removed the api label Sep 9, 2020

ping-sun requested a review from lizan September 9, 2020 16:14

introduce init_dump_handler_lib

8ef10ff

Signed-off-by: pingsun <pingsun@google.com>

ping-sun changed the title ~~API: Unready targets config dump~~ API: Unready Targets Dump Sep 9, 2020

remove unused using

9bd7dee

Signed-off-by: pingsun <pingsun@google.com>

lizan approved these changes Sep 10, 2020

View reviewed changes

lizan merged commit 8aef763 into envoyproxy:master Sep 10, 2020

Conversation

ping-sun commented Aug 7, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ping-sun commented Sep 4, 2020

Uh oh!

lambdai commented Sep 8, 2020

Uh oh!

ping-sun commented Sep 8, 2020

Uh oh!

htuch left a comment

Choose a reason for hiding this comment

Uh oh!

htuch commented Sep 9, 2020

Uh oh!

ping-sun commented Sep 9, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

ping-sun commented Aug 7, 2020 •

edited

Loading